Explainable Action Advising for Multi-Agent Reinforcement Learning

Action advising is a knowledge transfer technique for reinforcement learning based on the teacher-student paradigm. An expert teacher provides advice to a student during training in order to improve the student's sample efficiency and policy performance. Such advice is commonly given in the form of state-action pairs. However, it makes it difficult for the student to reason with and apply to novel states. We introduce Explainable Action Advising, in which the teacher provides action advice as well as associated explanations indicating why the action was chosen. This allows the student to self-reflect on what it has learned, enabling advice generalization and leading to improved sample efficiency and learning performance - even in environments where the teacher is sub-optimal. We empirically show that our framework is effective in both single-agent and multi-agent scenarios, yielding improved policy returns and convergence rates when compared to state-of-the-art methodsComment: This work has been accepted to ICRA 202

Nuovi laser a semiconduttore per sistemi di comunicazione ottici avanzati

Dottorato di ricerca in ingegneria elettronica. 9. ciclo. Tutore I. Montrosset. Coordinatore G. P. BavaConsiglio Nazionale delle Ricerche - Biblioteca Centrale - P.le Aldo Moro, 7, Rome; Biblioteca Nazionale Centrale - P.za Cavalleggeri, 1, Florence / CNR - Consiglio Nazionale delle RichercheSIGLEITItal

Tungsten-Based Nanocatalysts: Research Progress and Future Prospects

The high price of noble metal resources limits its commercial application and stimulates the potential for developing new catalysts that can replace noble metal catalysts. Tungsten-based catalysts have become the most important substitutes for noble metal catalysts because of their rich resources, friendly environment, rich valence and better adsorption enthalpy. However, some challenges still hinder the development of tungsten-based catalysts, such as limited catalytic activity, instability, difficult recovery, and so on. At present, the focus of tungsten-based catalyst research is to develop a satisfactory material with high catalytic performance, excellent stability and green environmental protection, mainly including tungsten atomic catalysts, tungsten metal nanocatalysts, tungsten-based compound nanocatalysts, and so on. In this work, we first present the research status of these tungsten-based catalysts with different sizes, existing forms, and chemical compositions, and further provide a basis for future perspectives on tungsten-based catalysts

Synthesis and Applications of SAPO-34 Molecular Sieves

Silicoaluminophosphate zeolite (SAPO-34) has been attracting increasing attention due to its excellent form selection and controllability in the chemical industry, as well as being one of the best industrial catalysts for methanol-to-olefin (MTO) reaction conversion. However, as a microporous molecular sieve, SAPO-34 easily generates carbon deposition and rapidly becomes inactivated. Therefore, it is necessary to reduce the crystal size of the zeolite or to introduce secondary macropores into the zeolite crystal to form a hierarchical structure in order to improve the catalytic effect. In this review, the synthesis methods of conventional SAPO-34 molecular sieves, hierarchical SAPO-34 molecular sieves and nanosized SAPO-34 molecular sieves are introduced, and the properties of the synthesized SAPO-34 molecular sieves are described, including the phase, morphology, pore structure, acid source, and catalytic performance, in particular with respect to the synthesis of hierarchical SAPO-34 molecular sieves. We hope that the review can provide guidance to the preparation of the SAPO-34 catalysts, and stimulate the future development of high-performance hierarchical SAPO-34 catalysts to meet the growing demands of the material and chemical industries

Table1_Relationship between lymph nodes examined and survival benefits with postoperative radiotherapy in oral cavity squamous cell carcinoma patients with stage T1-2N1M0.docx

BackgroundThis study aims to explore the relationship between the lymph nodes examined and survival benefits of postoperative radiotherapy in oral cavity squamous cell carcinoma patients with stage T1-2N1M0.MethodsThis study retrieved patients who underwent dissection of the primary site and neck lymph nodes for pT1-2N1M0 oral cavity squamous cell carcinoma without adverse nodal features from the Surveillance, Epidemiology, and End Results database from 2004 to 2015. Propensity score matching analysis was conducted, and the best cutoff value of the lymph nodes examined was determined by X-tile. Cancer-specific survival was the primary outcome. Univariable and multivariable analyses were performed to assess the relation between postoperative radiotherapy and cancer-specific survival, adjusting for other prognostic factors.ResultsA total of 469 patients were finally enrolled according to our exclusion criteria, and then 119 pairs of patients were matched by propensity score matching analysis. The best cutoff value of the lymph nodes examined was determined by X-tile, stratifying patients into lymph nodes examined ≤16 group and lymph nodes examined >16 group. For the whole matched cohort, the choice of postoperative radiotherapy had no correlation with other factors (all p’s > 0.05), and postoperative radiotherapy made no contribution to a better survival outcome for patients (p = 0.289). After stratified by the lymph nodes examined, in the lymph nodes examined ≤16 group, significantly improved CSS was found for those who undertook postoperative radiotherapy compared to those who just received surgery (unadjusted hazard ratio, 0.541; 95% confidence interval, 0.333–0.878; p = 0.013).ConclusionsOur study revealed that pT1-2N1M0 oral cavity squamous cell carcinoma patients were more likely to benefit from postoperative radiotherapy when unsatisfactory neck dissection was conducted, indicating that the number of lymph nodes examined might be a factor when clinicians do therapeutic planning for early-stage oral cavity squamous cell carcinoma patients.</p